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Developing Performance Measures for the 
Equity of Access and Student Success 



Introduction 

As postsecondary education becomes a necessity for nearly all U.S. residents to compete for 
living-wage jobs, the ability of colleges and universities to work collectively to expand access 
and the opportunity to succeed is vital to our economic well-being. With the exception of a 
relatively small set of institutions, institutional contributions to this national imperative 
hinges on the degree to which they “act locally”. Based on extensive research conducted in a 
variety of states, the vast majority of postsecondary institutions (two- and four-year) draw 
their undergraduates from geographic areas that fit within a relatively small radius around the 
institution. Despite this reality, many four-year institutions are reluctant to embrace the 
regional service concept for fear that their reputations and desires to become more selective 
might be threatened. The aspiration to become the best at serving their regions is rare, but 
sorely needed across the U.S. This is especially true in regions that are becoming more 
diverse - with growing racial/ ethnic populations that are underserved. 

NCHEMS has worked with the National Association of Systems heads (NASH) - with 
support from the Lumina Foundation for Education - to develop (1) empirically-based 
access regions for postsecondary institutions based on student enrollment patterns and (2) 
access, transition, and completion measures to gauge how well institutions serve their 
regions with respect to racial/ ethnic equity. This work was conducted to provide additional 
information to the existing comparative college graduation rates by institution and 
race/ ethnicity presented on the website www.collegeresults.org - developed by the 
Education Trust. From a combination of detailed student enrollment data provided by 
several states (by county of student residency), state-to-state student migration data provided 
by the National Center for Education Statistics (NCES), and additional data from NCES, we 
developed the following categories of four-year postsecondary institutions: 

• National Institutions . Highly-selective or special status independent institutions that 
have the potential to recruit on a national basis, an average entering ACT score of 28 
or above (or SAT equivalent), or HBCU status (e.g., Harvard, Stanford, Vanderbilt, 
etc.). 

• Statewide Institutions . Public institutions with a defined and recognized statewide 
mission including any of the following: a) “Flagship” or Land Grant status, 

b) membership in the Council of Public Liberal Arts Colleges (COPLAC), c) ACT 
score 26 or above, d) state HBCU status, e) state health sciences or engineering 
institution, or f) state military school. 



• Multi-State Institutions . Independent institutions that enroll heavily from outside the 
state in which they are located (but are not selective enough to be classified as 
National institutions), 45 percent or more undergraduate enrollment drawn from 
outside the state in which they are located, or ACT score 26 or better. The access 
region defined for these institutions is the state and all contiguous states. 

• Urban Institutions . Public universities that are predominantly commuter campuses, 
Carnegie Cl ass =15 and 16, and percent part-time undergraduate headcount greater 
than or equal to 20 percent. The county in which the institution is located is the 
access region. 

• Regional Institutions . Institutions that do not fit any of the above criteria and 
primarily serve students from sub-state regions within the state they are located. This 
is the largest category - representing nearly 80 percent of all four-year postsecondary 
institutions. 

The examples used throughout this report focus primarily on public four-year institutions 
that have (or should have) regional missions - those that fall into the later two categories 
above. The regional service mission of two-year colleges is obvious and all of the guidelines 
provided below can be easily applied to them as well. The analytical framework can also be 
applied to institutions that draw their students from an entire state, region, or that compete 
in the national market. It can also be applied to private institutions that fit into any of the 
above categories. 

The report provides a methodological framework for defining service regions and 
developing measures of performance with respect to the: 

• Equity of service with respect to certain race/ ethnic populations in their region, and 

• Persistence and completion rates of these students once they enter the institution. 

Drawing from NCHEMS’ recent work in Missouri, Kentucky, Ohio, and Tennessee, we 
provide a set of straightforward analytical steps for constructing indicators for each of the 
above in order to expand the tool kit for policy analysts and the overall policy framework for 
regional stewardship. 

Step 1: Defining Service Regions 

Over the past several years, NCHEMS has worked with state higher education policymakers 
in a variety of states to assist them in the development of better policy and practice around 
the issues of access and opportunity. In this process, we routinely examine student 
enrollment patterns - specifically where postsecondary institutions draw the majority of their 
students from. Simple tables like the one shown below (Figure 1) were provided to 
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NCHEMS by the state coordinating/ governing bodies for higher education or the 
institutions themselves. 



Figure 1 

County of Origin of First-Time Students Attending Kentucky 
Public Four-Year Institutions (Fall 2006) 



County of Origin of 
First-Time Students 


Eastern 

Kentucky 

University 


Kentucky 

State 

University 


Morehead 

State 

University 
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State 

University 
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Kentucky 

University 


University 

of 
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Allen County 
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1 


- 
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11 


- 


4 
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3 


Barren County 


1 


- 


2 


- 


- 


18 


7 


159 


Bath County 


3 


- 


53 


- 


- 


4 


- 


1 


Bell County 


15 


- 


2 


- 


- 


9 


3 


3 


Boone County 


62 


- 


26 


- 


288 


128 


64 


14 


Bourbon County 


23 


1 


6 


- 


- 


20 


1 


3 


Boyd County 


23 


2 


50 


1 


2 


37 


7 


1 


Boyle County 


48 


1 


3 


1 


3 


23 


5 


17 


Bracken County 


2 


- 


4 


- 


15 


5 


- 


- 


Breathitt County 


2 


- 


8 


- 


- 


4 


1 


- 


Breckinridge County 


2 


- 


- 


22 


1 


4 


9 


20 


Bullitt County 


10 


- 


7 


6 


2 


14 


71 


34 


Butler County 


2 


- 


- 


- 


- 


5 


3 


52 


Caldwell County 


2 


- 


- 


19 


- 


8 


1 


17 


Calloway County 


- 


- 


- 


156 


- 


12 


4 


2 


Campbell County 


36 


- 


14 


1 


292 


92 


35 


19 


Carlisle County 


- 


- 


1 


3 


- 


1 


- 


- 


Carroll County 


1 


- 


4 


- 


9 


9 


2 


1 


Carter County 


3 


- 


83 


- 


- 


5 


- 


- 


Casey County 


27 


- 


- 


- 


- 


6 


- 


4 


Christian County 


4 


9 


1 


49 


- 


28 


16 


36 


Clark County 


52 


- 


7 


2 


- 


43 


8 


10 



Source: Kentucky Council on Postsecondary Education 



Figure 1 shows the counties of origin of all first-time students attending the eight public 
universities in Kentucky (only 25 of 120 counties are included for illustration). These data 
for institutions in Kentucky and the same for institutions in Missouri, Ohio, and Tennessee 
reveal that most public four-year institutions are remarkably regional. The majority of their 
students (80%) come from a relatively small number of counties — clustered tightly around 
each of the institutions. The exceptions in these states are the flagship and land-grant 
institutions (such as the University of Tennessee at Knoxville, the University of Kentucky, 
Ohio State University, etc.) and more selective public institutions such as Truman State 
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University (MO), University of Missouri Rolla (MO), and Miami University (OH) — all of 
which have specific state-wide missions to serve a more academically prepared student body. 
For these institutions, the state is their service region as opposed to a selected number of 
counties. 

Once the data shown in Figure 1 are obtained, it is a fairly straight-forward exercise to 
determine which geographic areas an institution gets the majority its students from. In the 
case of Austin Peay State University below (Figure 2), we sorted the number of first-time 
students by county of origin - from high to low - and calculated a cumulative percentage in 
each county until we reached 80 percent of all first-time students. In this case, eight counties 
account for the majority of its first-time students. 
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Figure 2 

County of Origin of First-Time Students Attending 
Austin Peay State University (Fall 2001) 



County of Origin of 
First-Time Students 


Number of First-Time 
Students (in Descending 
Order) 


Cummualtive Percent 


Montgomery 


470 


51.0% 


Davidson 


72 


58.8% 


Robertson 


48 


64.1% 


Dickson 


47 


692% 


Cheatham 


36 


73.1% 


Shelby 


28 


76.1% 


Houston 


19 


782% 


Wilson 


18 


80.1% 


Sumner 


16 


81.9% 


Williamson 


16 


83.6% 


Stewart 


15 


852% 


Humphreys 


14 


86.8% 


Rutherford 


13 


882% 


Hamilton 


9 


89.1% 


Marshall 


8 


90.0% 


Knox 


6 


90.7% 



Source: Tennessee Higher Education Commission 



Once the service regions are defined, the data can be presented more effectively in a map 
that visually displays the defined service region - as shown in Figure 3. Austin Peay State 
University draws the vast majority of its students from nearby counties and a small 
percentage from Memphis, TN (Shelby County). 
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Figure 3 

Counties from which Austin Peay State University Receives 80 Percent of its 
First-Time Undergraduate Enrollment, Fall 2001 
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Montgomery = 51.0% 

Davidson = 7.8% 

Robertson = 5.2% 

Dickson = 5.1% 

Cheatham = 3.9% 

Other Service Counties = 1.9% to 3.0% 



Source: Tennessee Higher Education Commission. 



One could just as easily apply a more strict definition of a service region by restricting the 
cumulative percentage to 90 percent or (vice-versa) a less strict definition at 70 percent, for 
example. As seen above, the counties that combine to make up a service area do not need to 
be geographically contiguous. Many public four year institutions located in rural areas tend 
to draw their students from local areas and the nearest urban area because the local 
population base is not large enough to support the size of the institution. Another example 
(shown in Figure 4) is Southeast Missouri State University - where they draw the majority of 
their students locally and also from nearby St. Louis. 
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Figure 4 

Counties from which Southeast Missouri State University Receives 80 Percent of its 
First-Time Undergraduate Enrollment, Fall 2004 




Finally, different geographic boundaries of student residence can also be used to determine a 
service region - e.g., zip codes, census tracts, etc. This would require that these elements be 
included in the student record database and that access to the demographic information 
outlined in the following section be available. 

Step 2: Demographics of a Service Region 

In order to determine whether an institution is achieving equity in serving a region, it is also 
necessary to compile the race/ ethnic data for the target populations in the region. In our 
work with the four states, we identified (using data from the 2000 Census), the number and 
proportions of 1 8- to 24-year-olds with only a high school diploma by race/ ethnicity in each 
of the institutions’ service regions. We also identified the number and proportions of high 
school graduates by race/ ethnicity for the same regions (provided by the National Center for 
Education Statistics). Examples of these data for several institutions in Missouri are shown 
in Figure 5. 
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Figure 5 

Proportions of 18- to 24-Year-Olds with Just a High School Diploma and High School 
Graduates by Service Region (Selected Missouri Institutions) 





% 18-24 Year 


% High 


Institution 


Olds with Just 


School 




HS Diploma or 


Grads, Fall 




Equivalent, 2000 


2002 


Central Missouri State University 






White 


73.4 


77.6 


African-American 


19.2 


18.5 


Hispanic 


3.9 


1.7 


Lincoln University 






White 


58.2 


64.8 


African-American 


33.3 


30.8 


Hispanic 


4.3 


1.7 


Missouri Western State College 






White 


67.8 


71.8 


African-American 


23.9 


23.7 


Hispanic 


4.3 


20 


Northwest Missouri State University 






White 


65.7 


73.0 


African-American 


26.6 


23.0 


Hispanic 


3.8 


1.7 


Southeast Missouri State University 






White 


67.2 


74.3 


African-American 


27.3 


23.2 


Hispanic 


2.3 


0.6 



Source: U.S. Census Bureau; NCES Common Core Data 



By selecting these ages and the number of high school graduates, it is possible to establish a 
baseline for the race/ ethnic proportions of high school graduates and traditional-age college 
students (those theoretically ready to enter postsecondary education) - in order to determine 
whether these baseline proportions hold steady throughout subsequent stages of the 
education pipeline. 

For institutions that serve more non-traditional students (e.g., community colleges and four- 
year commuter institutions in urban areas), it might be desirable to expand the criteria to 
include older age-groups - e.g., 25- to 44-year-olds. The data for college participation and 
completion are discussed below. 



Step 3: College Participation and Completion 

Participation and completion data by institution and race/ ethnicity are available from the 
National Center for Education Statistics’ Integrated Postsecondary Education Data System 
(IPEDS). Staffs from postsecondary institutions that wish to conduct these analyses also 
have these data in their student unit record systems since they report these data to NCES. 

To complete the education pipeline concept, it is possible to extract cross-sectional 
race/ ethnic data for first-time freshmen, all other undergraduates, and degrees awarded from 
IPEDS. IPEDS also contains six-year graduation rates for full-time degree seeking students 
by race/ ethnicity. Together these data provide a good picture of the success of race/ ethnic 
populations at each of these stages in the education process. Figure 6 displays the 
proportions by race/ ethnicity for each of these data elements for a selected number of 
public four-year institutions in Missouri. 



Figure 6 

Participation, Completion, and Graduation Rates by Service Region 
(Selected Missouri Institutions) 





% First-Time 


% All Other 


% Bachelor's 


Six-Year 


Instutuion 


Freshmen, Fall 


Undergraduates, 


Degrees Awarded, 


Graduation Rates, 




2005 


Fall 2005 


2004-05 


2005 


Missouri Western State College 

White 


81.0 


87.9 


90.8 


30.7 


African-American 


15.6 


8.8 


6.0 


17.0 


Hispanic 


1.5 


1.6 


1.8 


16.7 


Northwest Missouri State University 

White 


91.7 


92.2 


94.0 


53.4 


African-American 


3.6 


2.5 


2.0 


34.5 


Hispanic 


1.8 


1.1 


0.8 


28.6 


Southeast Missouri State University 

White 


88.4 


89.9 


91.8 


50.8 


African-American 


8.6 


5.7 


4.3 


34.6 


Hispanic 


0.6 


0.8 


0.5 


30.0 


University of Missouri - Kansas City 

White 


63.8 


71.2 


69.5 


45.0 


African-American 


14.5 


11.1 


12.0 


30.5 


Hispanic 


3.3 


3.5 


3.2 


40.0 


University of Missouri - St. Louis 

White 


76.1 


81.8 


77.6 


39.5 


African-American 


14.6 


12.0 


9.8 


17.6 


Hispanic 


0.6 


1.4 


1.4 


NA 



Note: Hispanic graduation rates are based on small cohort sizes. 
Source: NCES: IPEDS Enrollment, Completions, and Graduation Rate Surveys 



These data are available annually from NCES. While not applicable in Missouri because of 
small numbers, data are also available for Asians and Native Americans. 
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Step 4: Piecing It All Together 

Once the institution service areas are defined and the demographic, participation, and 
completion data are compiled, it is possible to present the data in a way that shows how well 
institutions are serving their region with respect to racial/ ethnic equity. These data are 
displayed in Figure 7 for one of the public four-year institutions in Ohio. 



Figure 7 

Racial/Ethnic Representation of a Public Four-Year University’s Target Population, 
University Participation and Completion, and Graduation Rates 

B White B African-American d Hispanic 




High School 
Graduates, 2001- 
2002 (Service 
Area) 



% 18-24 Year 
Olds with Just a 
High School 
Diploma or 
Equivalent, 2000 
(Service Area) 



First-Time 
Freshman, Fall 
2005 (Institution) 



All Other 
Undeigraduates, 
FaH 2005 
(Institution) 



Bachelors Degrees 
Awarded 2004-05 
(Institution) 



Graduation Rates 
(Completers within 
150% of Program 
Time), 2005 
(Institution) 



Sources: NCES: Common Core Data; IPEDS Enrollment, Completions, and Graduation Rate Surveys, U.S. Census Bureau 



This university draws the majority (80%) of its students from seven counties. Within these 
counties, 80.5 percent of the high school graduates were White, 16.6 percent were African- 
American, and 1.3 percent Hispanic. Larger proportions of 18- to 24-year-olds with just a 
high school diploma were African-American and Hispanic (19.2 and 2.8 percent) - indicating 
that among the young adults who complete high school, minorities are less likely to enroll in 
postsecondary education. Both of these populations represent target populations for the 
university. 
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Relative to the target populations, the proportions of African-Americans and Hispanics who 
enrolled at the university as first-time freshmen drop fairly dramatically — to 8.9 percent 
African-American and 1.8 percent Hispanic. In each subsequent stage of academic progress 
within the institution, the proportion of Whites increase and the proportion of minorities 
decrease. Whites represent over 90 percent of the bachelor’s degrees produced compared to 
5.3 percent for African-Americans and less than one percent for Hispanics. The drop-off of 
minorities at each stage in the education process is confirmed by the graduation rates by 
race/ ethnicity - a cohort-based measure for graduation within six-years. All combined, 
these metrics suggest that the university could develop more successful programs geared to 
the minorities in its service region - at each stage of the education process (participation, 
persistence, and completion). 

Supplemental Data and Information 

For the institutions that have fairly distinct sub-state service regions, it is helpful to compile 
additional data that yield a much greater understanding of the demographic and economic 
conditions of the region relative to other parts of the state. Some of the most useful data 
include: 

• The educational attainment of adults by age and degree-level - by race/ ethnicity 
where appropriate 

• Employment by Occupation and Industry - an indication of the types of jobs 
available in the region 

• Numbers and percentage of adults with a high school diploma or less, living in 
families that earn less than a living wage - an indication of the size of the adult 
population that needs postsecondary education and training in order to advance their 
life circumstances. 

• Local migration patterns by degree-level and occupation - indicating the strength of 
the local economy and the ability of the region to retain or attract high-skilled 
workers. 

• Annual earnings by degree-level and the difference in earnings between a high school 
diploma and a college degree - another indicator of the strength of the local 
economy, as well as the monetary return that individuals experience as a result of 
college education. 

An example of these data and information for a region in Oklahoma is included in the 
appendix. Much of these data are available from the U.S. Census Bureau’s annual American 
Community Survey (ACS), and even more detailed data can be extracted from the ACS 
Public Use Microdata Samples. These data - in combination with the data on equity, access 
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and success - provide a robust picture of the environment in which institutions operate and 
their ability to act locally. 

Conclusion 

With the exception of the student county-of-origin data provided by the State Higher 
Education Executive Offices (SHEEOs), the data used throughout these calculational 
procedures are readily available from public sources. Provided access to student unit record 
data at the state or institutional levels, it may be possible to improve or expand this work by: 

• Actually tracking students (by race/ ethnicity) through the education system rather 
than relying on cross-sectional metrics. 

• Developing a similar analytical process for low-income students. From public data 
sources, it is possible to establish how many low-income residents there are in 
geographic regions, but not how many enroll in, and complete, postsecondary 
education. If institutions or state agencies collect data on levels of student or family 
income, and have the ability to track them through the education system, it is possible 
to base these analyses on income in addition to race/ ethnicity. Because 

student/ family income data are usually only available for those who apply for 
financial aid, comprehensive studies of this nature are rare. 

These data and information suggest the potential utility of using cross-sectional statistics to 
estimate equity in college access and success. They can also be used for benchmarking 
purposes in much the same way as the Education Trust’s Lumina project currently 
benchmarks graduation rates based on the National Center for Education Statistics’ 
Graduation Rate Survey (GRS). Finally, they also illustrate the parallel potential of using 
cross-sectional “pipeline-type” estimates to supplement cohort-based statistics drawn from 
the GRS. While GRS statistics are superior to cross-sectional estimates from a purely 
methodological standpoint because the data are truly longitudinal, they are limited to full- 
time, first-time students and do not represent most students at many colleges and 
universities. Cohorts can also be limited by small numbers and do not provide information 
about access for underserved populations. The cross-sectional approach provides a useful 
supplement to the GRS. 
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Data Sources 



Student County of Origin Data: Kentucky Council on Postsecondary Education, Missouri 

Department of Higher Education, Ohio Board of Regents, and Tennessee Higher Education 
Commission. 

High School Graduates by Race/Ethnicity: National Center for Education Statistics, Common Core 
Data. 

18- to 24-Year-Olds with Just a High School Diploma by County and Race/Ethnicity: U.S. Census 
Bureau, 2000 Decennial Census. 

First-Time Students by Race/Ethnicity: National Center for Education Statistics, IPEDS Fall 
Enrollment Survey. 

All Other Undergraduates by Race/Ethnicity: National Center for Education Statistics, IPEDS Fall 
Enrollment Survey. 

Degrees Awarded by Race/Ethnicity: National Center for Education Statistics, IPEDS Completions 
Survey. 

Six-Year Graduation Rates of Bachelor’s Students: National Center for Education Statistics, IPEDS 
Graduation Rate Survey. 
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Appendix 1 — Graphs and Charts 

Sources: U.S. Census Bureau, 2005-07 American Community Survey (ACS) Three-Year Public Use 
Microdata Sample (PUMS) File. 
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Age 18 to 24 Population Change by Region, 2000-07 
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Age 25 to 49 Population Change by Region, 2000-07 
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Educational Attainment of 18 to 24 Year Olds, 2005-07 
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Educational Attainment (%) of 25 to 64 Year Olds by Race/Ethnicity, 

2005-07 (Northwest/Panhandle) 
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Percent of 25 to 64 Year Olds with a High School Diploma, but No 

College, 2005-07 
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Percent of 25 to 64 Year Olds with a Bachelor’s Degree or Higher, 

2005-07 
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Percent of 18 to 24 Year Olds with a High School Diploma, But No 

College, 2005-07 



South Central/Lawton 
Guthrie/Shawnee 
Southeast 
Eastern 
Northeast 
Rogers/Waggoner 
El Reno/Chickasha 
Enid/Ponca City 
Southern 
Oklahoma 
Northwest/Panhandle 
Southwest 
U.S. 

TulsaCounty 
OklahomaCity Metro 
Osage/Stillwater/Seminole 
Norman/Moore 




0 10 20 30 40 50 60 



Percent of 18 to 64 Year Olds Who Have Just a High School Diploma 
or Less and are Living in Families with Incomes Below a Living Wage 



Eastern 
Southeast 
Southwest 
Northeast 
Southern 
Enid/Ponca City 
Guthrie/Shawnee 
South Central/Lawton 
Northwest/Panhandle 
Osage/Stillwater/Seminole 
Oklahoma 
OklahomaCity Metro 
TulsaCounty 
U.S. 

El Reno/Chickasha 
Rogers/Waggoner 
Norman/Moore 




0 5 10 15 20 25 30 



20 




Number of 18 to 64 Year Olds Who Have Just a High School Diploma or 
Less and are Living in Families with Incomes Below a Living Wage 
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Percentage Employment by Occupation, 2005-07 

□ U.S. c Oklahoma ■Northwest/Panhandle 
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Average Annual Net Migration by Degree-Level (2005-2007) 22-64 Year 
Olds (Northwest /Panhandle Region) 
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